Serotonin, in its function as neurotransmitter, is well-known for its role in depression, autism and other neuropsychiatric disorders, however, less known as a neurodevelopmental factor. The serotonergic system is one of the earliest to develop during embryogenesis and early changes in serotonin levels can have large consequences for the correct development of specific brain areas. The regulation and functioning of serotonin is influenced by genetic risk factors, such as the serotonin transporter polymorphism in humans. This polymorphism is associated with anxiety-related symptoms, changes in social behavior, and cortical gray and white matter changes also seen in patients suffering from autism spectrum disorders (ASD). The human polymorphism can be mimicked by the knockout of the serotonin transporter in rodents, which are as a model system therefore vital to explore the precise neurobiological mechanisms. Moreover, there are pharmacological challenges influencing serotonin in early life, like prenatal/neonatal exposure to selective serotonin reuptake inhibitors (SSRI) in depressed pregnant women. There is accumulating evidence that this dysregulation of serotonin during critical phases of brain development can lead to ASD-related symptoms in children, and reduced social behavior and increased anxiety in rodents. Furthermore, prenatal valproic acid (VPA) exposure, a mood stabilizing drug which is also thought to interfere with serotonin levels, has the potency to induce ASD-like symptoms and to affect the development of the serotonergic system. Here, we review and compare the neurodevelopmental and behavioral consequences of serotonin transporter gene variation, and prenatal SSRI and VPA exposure in the context of ASD.